Moodle survey

We are importing the data from the students answers in 5 years, between 2018-2022. The quiz consists on up to 77 questions, being 5 free-text questions (50, 51, 52, 75, 76 & 77), 7 three-options questions and the rest in Likert scale. The latter will be categorized as “Positive” or “Negative” if 5 (“Totally agree”) means a “good” or “bad” feedback.

#load questions
questions <- read.csv("questions.csv", head=TRUE, sep=";")
questions <- cbind(row.names(questions),questions[,c(3,1,2)])
#add type variable
questions$type <- "Pos."
questions$type[questions$Section=="Open"] <- ""
questions[c(4,10,56,63,63,64,65,66,67,72),5] <- "Neg."
questions$type[questions$Section=="Open"] <- "NA"
colnames(questions) <- c("No.","Since","Section","Question","Type")

kbl(questions, align = "cccl", caption = "Table 1. Students opinion quizz. The bulk of the questionaire was designed for the year 2017 and new questions were added as indicated in the rightmost column.") %>%
    kable_styling(bootstrap_options = "striped", full_width = F) %>%
    column_spec(1, italic = T)
Table 1. Students opinion quizz. The bulk of the questionaire was designed for the year 2017 and new questions were added as indicated in the rightmost column.
No.  Since Section Question Type
1 2017-2018 Equipment We had sufficient amount of small lab equipment (e.g. pippettes, cuvettes,…) Pos.
2 2017-2018 Equipment Access to general instruments/equipment (e.g. PCR, laminar flow hoods) was limiting Pos.
3 2017-2018 Equipment Lab equipment was well mantained/modern Pos.
4 2017-2018 Length and schedule The course is too long Neg.
5 2017-2018 Length and schedule The period allocated for the course within the term is appropriate Pos.
6 2017-2018 Length and schedule On average, the number of hours per day is sufficient Pos.
7 2017-2018 General Methodology Student engagement Pos.
8 2017-2018 General Methodology Course interest Pos.
9 2017-2018 General Methodology Course usefulness Pos.
10 2017-2018 General Methodology Demand on student’s part Neg.
11 2017-2018 General Methodology Course difficulty Pos.
12 2017-2018 Method objectives State a hypothesis Pos.
13 2017-2018 Method objectives Design an experimental strategy to answer specific biological questions Pos.
14 2017-2018 Method objectives Identify cost-effective (time and money-wise) practical techniques to answer a given question Pos.
15 2017-2018 Method objectives Write an experimental protocol including all relevant controls Pos.
16 2017-2018 Method objectives Execute an experimental protocol Pos.
17 2017-2018 Method objectives Record, analyze and interpret experimental results Pos.
18 2017-2018 Method objectives Integrate information from other researchers and discuss your results within a larger context Pos.
19 2017-2018 Method objectives Follow lab safety rules Pos.
20 2017-2018 Method objectives Ability to work in a team Pos.
21 2017-2018 Activities interest Introductory lecture (day 1) Pos.
22 2017-2018 Activities interest Stats Intro (day 2) Pos.
23 2017-2018 Activities interest Initial brainstorming session (day 2) Pos.
24 2017-2018 Activities interest Group meetings (experiment planning and design) Pos.
25 2017-2018 Activities interest Team feed-back session (propossal design) Pos.
26 2017-2018 Activities interest Final group discussion session (day before the last) Pos.
27 2017-2018 Activities interest Final class discussion session (last day) Pos.
28 2017-2018 Activities Length Introductory lecture (day 1) Pos.
29 2017-2018 Activities Length Stats Intro (day 2) Pos.
30 2017-2018 Activities Length Initial brainstorming session (day 2) Pos.
31 2017-2018 Activities Length Group meetings (experiment planning and design) Pos.
32 2017-2018 Activities Length Team feed-back sessions Pos.
33 2017-2018 Activities Length Final group discussion session (day before the last) Pos.
34 2017-2018 Activities Length Final class discussion session (last day) Pos.
35 2017-2018 Assesment Safety and clean-up weekly evaluation Pos.
36 2017-2018 Assesment Individual lab notebook evaluation (design) Pos.
37 2017-2018 Assesment Group lab notebook evaluation (design) Pos.
38 2017-2018 Assesment Group lab notebook evaluation (results) Pos.
39 2017-2018 Assesment Peer-evaluation (Team) Pos.
40 2017-2018 Assesment Final exam Pos.
41 2017-2018 Learning Objectives State a hypothesis Pos.
42 2017-2018 Learning Objectives Design an experimental strategy to answer specific biological questions Pos.
43 2017-2018 Learning Objectives Identify cost-effective (time and money-wise) practical techniques to answer a given question Pos.
44 2017-2018 Learning Objectives Write an experimental protocol including all relevant controls Pos.
45 2017-2018 Learning Objectives Execute an experimental protocol Pos.
46 2017-2018 Learning Objectives Record, analyze and interpret experimental results Pos.
47 2017-2018 Learning Objectives Integrate information from other researchers and discuss your results within a larger context Pos.
48 2017-2018 Learning Objectives Follow lab safety rules Pos.
49 2017-2018 Learning Objectives Ability to work in a team Pos.
50 2017-2018 Open Best NA
51 2017-2018 Open Wors NA
52 2019-2020 Open Open comments NA
53 2019-2020 ELN I knew OneNote before the course. Pos.
54 2019-2020 ELN I find OneNote a useful app for the elaboration of the course Notebook. Pos.
55 2019-2020 ELN I think we had enough previous information about the use of OneNote for the course laboratory notebook. Pos.
56 2019-2020 ELN I would prefer other applications like Evernote, Google Docs, or MS Word. Neg.
57 2019-2020 ELN The structure of the Notebook in Sections, Pages, and Subpages was easy to understand and use. Pos.
58 2019-2020 ELN I would recommend to use OneNote for the ELN next year. Pos.
59 2019-2020 ELN It overall allowed us to save time through the whole course. Pos.
60 2019-2020 ELN It helped us in the preparation of the final version to be presented. Pos.
61 2019-2020 ELN It made it easier working in groups. Pos.
62 2019-2020 ELN It could be useful for other practical courses. Pos.
63 2019-2020 ELN Synchronization issues hindered our teamworking. Neg.
64 2019-2020 ELN Conflicts of versions were very frequent. Neg.
65 2019-2020 ELN Access to the ELN was slow. Neg.
66 2019-2020 ELN We missed several wordprocessing features. Neg.
67 2019-2020 ELN Some features of the application did not work properly. Neg.
68 2019-2020 ELN The template helped us to prepare the final version of the notebook. Pos.
69 2019-2020 ELN The structure of the template matches the evaluation rubric. Pos.
70 2019-2020 ELN The structure of 4 independent notes helped to understand each step of the work in a research lab. Pos.
71 2019-2020 ELN The word limit in some sections of the template is appropriate (1 Pos.
72 2019-2020 ELN We had to modify some sections of the template to suit our experimental work. Neg.
73 2019-2020 ELN The template will facilitate an even evaluation. Pos.
74 2019-2020 ELN The use of an ELN with OneNote in this course will help us in future laboratory work (TFG, TFM, etc.). Pos.
75 2021-2022 Open Open comments ELN NA
76 2021-2022 Open Positive aspects about the project desing teams session NA
77 2021-2022 Open Negative aspects about the project desing teams session NA

Load data and pairwise t.test

Survey answers were downloaded from Moodle as txt/csv files. Moodle updates generated some format differences that could be bypassed after opening the files with Numbers and exporting as tables with “;” as column separator. We now show pairwise t.tests carried out to identify significant differences per year in the answers.

#read the data in a list of dataframes
#didn't use the headers to avoid mistakes
quiz <- lapply(2017:2022, function(x) read.csv(paste0("survey",x,".csv"),header=FALSE,skip=1,sep=";"))
#add Year as the third variable (empty so far)
curso <- c("2017","2018","2019","2020","2021","2022")
for (i in 1:length(quiz)){
  quiz[[i]][,3] <- curso[i]
}

#adjust questions changes
#remove questions in column 32 & 40 from 2017, because we removed it in the following years
quiz[[1]] <- quiz[[1]][,-c(32,40)]
quiz[[6]] <- quiz[[6]][,-10]
names(quiz[[6]]) <- names(quiz[[5]])
names(quiz[[1]]) <- names(quiz[[2]])
#merge all dataframes and name the columns 
data <- Reduce(function(x, y) merge(x, y, all=TRUE), quiz)
#take the colnames from the last quiz that contains all the questions

colnames(data)[3] <- "Curso"


#statistics analysis
#subset questions 1: remove leftmost junk columns
subdata <- data[,c(3,11:87)]
names(subdata) <- c("Curso",paste0("Q",1:77))
#subset questions 2: remove open questions
open <- c(row.names(questions[questions$Section=="Open",]))
subdata <- subdata[,-(as.integer(open)+1)]
subdata <- sapply(subdata,as.numeric)
subdata <- as.data.frame(subdata)


tests <- list()
nombres <- c()

for (i in 2:ncol(subdata)){
  subdata[,i][!(subdata[,i] %in% c(1,2,3,4,5))] <- NA
  #subset for years with answers to avoid void groups
  kkk <- subset(subdata,!is.na(subdata[,i]))
  tests[[i-1]] <- pairwise.t.test(x=as.numeric(kkk[,i]),g=as.numeric(kkk[,1]),paired = F)
  nombres[i-1] <- questions[,4][(which(questions$No. %in% gsub("\\D", "",colnames(subdata[i]))))]
  names(tests) <- nombres
  print(names(tests[i-1]))
  print(tests[[i-1]]$p.value)
}
## [1] "We had sufficient amount of small lab equipment (e.g. pippettes, cuvettes,...)"
##              2017       2018      2019        2020 2021
## 2018 4.835902e-05         NA        NA          NA   NA
## 2019 3.879170e-02 0.29237424        NA          NA   NA
## 2020 4.884986e-07 1.00000000 0.0419191          NA   NA
## 2021 2.620952e-01 0.05789424 1.0000000 0.003876997   NA
## 2022 4.191910e-02 0.45476018 1.0000000 0.092183903    1
## [1] "Access to general instruments/equipment (e.g. PCR, laminar flow hoods)  was limiting"
##           2017 2018 2019      2020 2021
## 2018 1.0000000   NA   NA        NA   NA
## 2019 1.0000000    1   NA        NA   NA
## 2020 0.8109768    1    1        NA   NA
## 2021 1.0000000    1    1 0.8344372   NA
## 2022 1.0000000    1    1 1.0000000    1
## [1] "Lab equipment was well mantained/modern"
##           2017 2018 2019 2020 2021
## 2018 0.4899485   NA   NA   NA   NA
## 2019 1.0000000    1   NA   NA   NA
## 2020 1.0000000    1    1   NA   NA
## 2021 1.0000000    1    1    1   NA
## 2022 0.6531043    1    1    1    1
## [1] "The course is too long"
##              2017 2018 2019 2020 2021
## 2018 0.0026643833   NA   NA   NA   NA
## 2019 0.0004996605    1   NA   NA   NA
## 2020 0.1019667926    1    1   NA   NA
## 2021 0.0868213996    1    1    1   NA
## 2022 0.1248722514    1    1    1    1
## [1] "The period allocated for the course within the term is appropriate"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "On average, the number of hours per day is sufficient"
##              2017         2018        2019         2020      2021
## 2018 0.0001740842           NA          NA           NA        NA
## 2019 0.7083631075 0.0082785316          NA           NA        NA
## 2020 1.0000000000 0.0000373255 0.557747775           NA        NA
## 2021 0.1093921851 0.2837206178 0.699933496 3.884956e-02        NA
## 2022 0.0002382551 1.0000000000 0.009910691 5.433618e-05 0.2837206
## [1] "Student engagement"
##      2017      2018      2019      2020 2021
## 2018    1        NA        NA        NA   NA
## 2019    1 1.0000000        NA        NA   NA
## 2020    1 0.2189352 0.3199053        NA   NA
## 2021    1 1.0000000 1.0000000 1.0000000   NA
## 2022    1 1.0000000 1.0000000 0.1269859    1
## [1] "Course interest"
##           2017 2018 2019      2020 2021
## 2018 1.0000000   NA   NA        NA   NA
## 2019 1.0000000    1   NA        NA   NA
## 2020 0.2988613    1    1        NA   NA
## 2021 1.0000000    1    1 0.1231708   NA
## 2022 1.0000000    1    1 0.3158588    1
## [1] "Course usefulness"
##      2017      2018     2019      2020 2021
## 2018    1        NA       NA        NA   NA
## 2019    1 1.0000000       NA        NA   NA
## 2020    1 0.3070813 0.401915        NA   NA
## 2021    1 1.0000000 1.000000 0.2706641   NA
## 2022    1 1.0000000 1.000000 1.0000000    1
## [1] "Demand on student's part"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "Course difficulty"
##           2017      2018     2019       2020 2021
## 2018 1.0000000        NA       NA         NA   NA
## 2019 1.0000000 1.0000000       NA         NA   NA
## 2020 0.5995961 0.2087174 0.752361         NA   NA
## 2021 1.0000000 1.0000000 1.000000 1.00000000   NA
## 2022 1.0000000 1.0000000 1.000000 0.09488378    1
## [1] "State a hypothesis"
##      2017 2018 2019      2020 2021
## 2018    1   NA   NA        NA   NA
## 2019    1    1   NA        NA   NA
## 2020    1    1    1        NA   NA
## 2021    1    1    1 0.5792492   NA
## 2022    1    1    1 1.0000000    1
## [1] "Design an experimental strategy to answer specific biological questions"
##           2017 2018 2019 2020 2021
## 2018 0.9819431   NA   NA   NA   NA
## 2019 1.0000000    1   NA   NA   NA
## 2020 1.0000000    1    1   NA   NA
## 2021 1.0000000    1    1    1   NA
## 2022 0.9819431    1    1    1    1
## [1] "Identify cost-effective (time and money-wise) practical techniques to answer a given question"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "Write an experimental protocol including all relevant controls"
##      2017 2018      2019 2020 2021
## 2018    1   NA        NA   NA   NA
## 2019    1    1        NA   NA   NA
## 2020    1    1 0.8385782   NA   NA
## 2021    1    1 0.9188235    1   NA
## 2022    1    1 1.0000000    1    1
## [1] "Execute an experimental protocol"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "Record, analyze and interpret experimental results"
##      2017      2018 2019       2020 2021
## 2018    1        NA   NA         NA   NA
## 2019    1 1.0000000   NA         NA   NA
## 2020    1 0.6303827    1         NA   NA
## 2021    1 1.0000000    1 0.67234486   NA
## 2022    1 1.0000000    1 0.07443221    1
## [1] "Integrate information from other researchers and discuss your results within a larger context"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "Follow lab safety rules"
##      2017      2018 2019 2020 2021
## 2018    1        NA   NA   NA   NA
## 2019    1 0.1591204   NA   NA   NA
## 2020    1 0.1062739    1   NA   NA
## 2021    1 1.0000000    1    1   NA
## 2022    1 1.0000000    1    1    1
## [1] "Ability to work in a team"
##           2017 2018      2019 2020 2021
## 2018 1.0000000   NA        NA   NA   NA
## 2019 0.1609205    1        NA   NA   NA
## 2020 1.0000000    1 0.7693568   NA   NA
## 2021 1.0000000    1 0.9866925    1   NA
## 2022 0.4842893    1 1.0000000    1    1
## [1] "Introductory lecture (day 1)"
##           2017      2018      2019 2020      2021
## 2018 1.0000000        NA        NA   NA        NA
## 2019 1.0000000 1.0000000        NA   NA        NA
## 2020 1.0000000 1.0000000 1.0000000   NA        NA
## 2021 1.0000000 1.0000000 1.0000000    1        NA
## 2022 0.1719199 0.2035414 0.2110407    1 0.4244263
## [1] "Stats Intro (day 2)"
##              2017      2018 2019 2020 2021
## 2018 9.016649e-02        NA   NA   NA   NA
## 2019 8.152269e-05 0.7431196   NA   NA   NA
## 2020 1.632072e-04 0.7465066    1   NA   NA
## 2021 8.674063e-05 0.7431196    1    1   NA
## 2022 8.588806e-04 1.0000000    1    1    1
## [1] "Initial brainstorming session (day 2)"
##           2017       2018      2019 2020 2021
## 2018 0.7350214         NA        NA   NA   NA
## 2019 1.0000000 1.00000000        NA   NA   NA
## 2020 1.0000000 0.54883014 1.0000000   NA   NA
## 2021 1.0000000 0.08910869 1.0000000    1   NA
## 2022 1.0000000 0.02113266 0.4213264    1    1
## [1] "Group meetings (experiment planning and design)"
##      2017 2018      2019      2020 2021
## 2018    1   NA        NA        NA   NA
## 2019    1    1        NA        NA   NA
## 2020    1    1 1.0000000        NA   NA
## 2021    1    1 1.0000000 1.0000000   NA
## 2022    1    1 0.4134602 0.4928046    1
## [1] "Team feed-back session (propossal design)"
##              2017         2018         2019         2020 2021
## 2018 1.000000e+00           NA           NA           NA   NA
## 2019 1.000000e+00 1.000000e+00           NA           NA   NA
## 2020 1.000000e+00 1.000000e+00 1.000000e+00           NA   NA
## 2021 5.122069e-05 9.484981e-08 1.323700e-06 9.133855e-08   NA
## 2022 1.580189e-03 9.221399e-06 8.708906e-05 9.221399e-06    1
## [1] "Final group discussion session (day before the last)"
##      2017      2018      2019 2020 2021
## 2018    1        NA        NA   NA   NA
## 2019    1 1.0000000        NA   NA   NA
## 2020    1 1.0000000 1.0000000   NA   NA
## 2021    1 0.8147184 0.8058628    1   NA
## 2022    1 1.0000000 1.0000000    1    1
## [1] "Final class discussion session (last day)"
##            2017 2018       2019 2020 2021
## 2018 1.00000000   NA         NA   NA   NA
## 2019 0.05752352    1         NA   NA   NA
## 2020 1.00000000    1 0.05752352   NA   NA
## 2021 1.00000000    1 0.19394869    1   NA
## 2022 1.00000000    1 0.13401895    1    1
## [1] "Introductory lecture (day 1)"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "Stats Intro (day 2)"
##              2017     2018      2019 2020      2021
## 2018 7.920217e-05       NA        NA   NA        NA
## 2019 1.121074e-02 0.923398        NA   NA        NA
## 2020 4.812777e-05 1.000000 0.9231893   NA        NA
## 2021 4.677896e-03 1.000000 1.0000000    1        NA
## 2022 5.872309e-06 1.000000 0.3164371    1 0.6849254
## [1] "Initial brainstorming session (day 2)"
##      2017      2018 2019 2020 2021
## 2018    1        NA   NA   NA   NA
## 2019    1 1.0000000   NA   NA   NA
## 2020    1 1.0000000    1   NA   NA
## 2021    1 0.9746626    1    1   NA
## 2022    1 0.4920238    1    1    1
## [1] "Group meetings (experiment planning and design)"
##      2017      2018 2019      2020      2021
## 2018    1        NA   NA        NA        NA
## 2019    1 0.5996380   NA        NA        NA
## 2020    1 1.0000000    1        NA        NA
## 2021    1 1.0000000    1 1.0000000        NA
## 2022    1 0.1165861    1 0.4314979 0.9808853
## [1] "Team feed-back sessions"
##              2017         2018         2019         2020      2021
## 2018 1.000000e+00           NA           NA           NA        NA
## 2019 1.000000e+00 1.000000e+00           NA           NA        NA
## 2020 1.000000e+00 1.000000e+00 1.000000e+00           NA        NA
## 2021 3.883002e-05 2.066388e-04 2.745649e-03 2.745649e-03        NA
## 2022 9.589759e-09 9.147903e-08 1.842478e-06 2.020658e-06 0.4156231
## [1] "Final group discussion session (day before the last)"
##           2017 2018      2019      2020 2021
## 2018 1.0000000   NA        NA        NA   NA
## 2019 1.0000000    1        NA        NA   NA
## 2020 1.0000000    1 1.0000000        NA   NA
## 2021 1.0000000    1 0.5053068 0.8400922   NA
## 2022 0.9660558    1 0.4379554 0.7278093    1
## [1] "Final class discussion session (last day)"
##            2017       2018       2019 2020 2021
## 2018 0.01265194         NA         NA   NA   NA
## 2019 0.12572353 1.00000000         NA   NA   NA
## 2020 1.00000000 0.00403720 0.04786672   NA   NA
## 2021 1.00000000 0.02036425 0.18483993    1   NA
## 2022 1.00000000 0.22708031 1.00000000    1    1
## [1] "Safety and clean-up weekly evaluation"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "Individual lab notebook evaluation (design)"
##              2017      2018 2019 2020 2021
## 2018 5.986044e-06        NA   NA   NA   NA
## 2019 3.624199e-02 0.1515468   NA   NA   NA
## 2020 3.708189e-03 0.9621303    1   NA   NA
## 2021 4.132923e-03 0.8032681    1    1   NA
## 2022 2.823163e-03 1.0000000    1    1    1
## [1] "Group lab notebook evaluation (design)"
##             2017      2018 2019 2020 2021
## 2018 0.009839814        NA   NA   NA   NA
## 2019 0.030480248 1.0000000   NA   NA   NA
## 2020 0.326865132 1.0000000    1   NA   NA
## 2021 0.862805649 0.8628056    1    1   NA
## 2022 1.000000000 0.5820280    1    1    1
## [1] "Group lab notebook evaluation (results)"
##           2017       2018      2019 2020 2021
## 2018 0.7761493         NA        NA   NA   NA
## 2019 1.0000000 0.85014326        NA   NA   NA
## 2020 1.0000000 0.08987183 1.0000000   NA   NA
## 2021 1.0000000 0.11606382 1.0000000    1   NA
## 2022 1.0000000 0.02078403 0.9301167    1    1
## [1] "Peer-evaluation (Team)"
##           2017      2018 2019 2020 2021
## 2018 0.2010087        NA   NA   NA   NA
## 2019 0.9374738 1.0000000   NA   NA   NA
## 2020 1.0000000 1.0000000    1   NA   NA
## 2021 1.0000000 0.4252861    1    1   NA
## 2022 1.0000000 1.0000000    1    1    1
## [1] "Final exam"
##            2017      2018        2019       2020         2021
## 2018 1.00000000        NA          NA         NA           NA
## 2019 1.00000000 1.0000000          NA         NA           NA
## 2020 1.00000000 1.0000000 1.000000000         NA           NA
## 2021 0.19443595 0.1119257 0.588829367 0.19443595           NA
## 2022 0.04751644 0.2425330 0.008068794 0.06184887 1.291026e-05
## [1] "State a hypothesis"
##            2017       2018      2019      2020 2021
## 2018 1.00000000         NA        NA        NA   NA
## 2019 0.04850727 0.04850727        NA        NA   NA
## 2020 1.00000000 1.00000000 0.2405459        NA   NA
## 2021 0.17406452 0.17406452 1.0000000 0.6417639   NA
## 2022 0.10429512 0.10429512 1.0000000 0.3946902    1
## [1] "Design an experimental strategy to answer specific biological questions"
##           2017       2018 2019 2020      2021
## 2018 0.2256925         NA   NA   NA        NA
## 2019 1.0000000 1.00000000   NA   NA        NA
## 2020 1.0000000 0.11316070    1   NA        NA
## 2021 1.0000000 0.05348894    1    1        NA
## 2022 1.0000000 1.00000000    1    1 0.6999663
## [1] "Identify cost-effective (time and money-wise) practical techniques to answer a given question"
##           2017      2018      2019 2020 2021
## 2018 0.8223218        NA        NA   NA   NA
## 2019 1.0000000 0.1174194        NA   NA   NA
## 2020 1.0000000 0.8007973 1.0000000   NA   NA
## 2021 1.0000000 1.0000000 0.8007973    1   NA
## 2022 1.0000000 1.0000000 1.0000000    1    1
## [1] "Write an experimental protocol including all relevant controls"
##            2017       2018 2019 2020 2021
## 2018 0.05017013         NA   NA   NA   NA
## 2019 1.00000000 0.11678843   NA   NA   NA
## 2020 1.00000000 0.03687698    1   NA   NA
## 2021 1.00000000 0.22014254    1    1   NA
## 2022 1.00000000 0.71000111    1    1    1
## [1] "Execute an experimental protocol"
##            2017      2018 2019 2020 2021
## 2018 0.06715688        NA   NA   NA   NA
## 2019 1.00000000 1.0000000   NA   NA   NA
## 2020 1.00000000 0.5912804    1   NA   NA
## 2021 1.00000000 0.2934492    1    1   NA
## 2022 1.00000000 0.5927727    1    1    1
## [1] "Record, analyze and interpret experimental results"
##            2017       2018 2019      2020 2021
## 2018 0.05965194         NA   NA        NA   NA
## 2019 1.00000000 0.17943892   NA        NA   NA
## 2020 1.00000000 0.08604197    1        NA   NA
## 2021 1.00000000 0.38331922    1 1.0000000   NA
## 2022 0.57856745 1.00000000    1 0.7137619    1
## [1] "Integrate information from other researchers and discuss your results within a larger context"
##      2017 2018 2019 2020 2021
## 2018    1   NA   NA   NA   NA
## 2019    1    1   NA   NA   NA
## 2020    1    1    1   NA   NA
## 2021    1    1    1    1   NA
## 2022    1    1    1    1    1
## [1] "Follow lab safety rules"
##      2017      2018 2019 2020 2021
## 2018    1        NA   NA   NA   NA
## 2019    1 0.3584685   NA   NA   NA
## 2020    1 1.0000000    1   NA   NA
## 2021    1 1.0000000    1    1   NA
## 2022    1 0.0763971    1    1    1
## [1] "Ability to work in a team"
##           2017      2018 2019 2020 2021
## 2018 0.5300693        NA   NA   NA   NA
## 2019 1.0000000 1.0000000   NA   NA   NA
## 2020 1.0000000 0.9486344    1   NA   NA
## 2021 1.0000000 1.0000000    1    1   NA
## 2022 1.0000000 1.0000000    1    1    1
## [1] "I knew OneNote before the course."
##             2019       2020       2021
## 2020 1.000000000         NA         NA
## 2021 1.000000000 1.00000000         NA
## 2022 0.001084163 0.01673493 0.01673493
## [1] "I find OneNote a useful app for the elaboration of the course Notebook."
##           2019         2020       2021
## 2020 0.0194452           NA         NA
## 2021 0.4290317 0.4290317085         NA
## 2022 0.4290317 0.0009279188 0.05334355
## [1] "I think we had enough previous information about the use of OneNote for the course laboratory notebook."
##           2019       2020      2021
## 2020 0.9032147         NA        NA
## 2021 0.2647527 0.08479041        NA
## 2022 0.2647527 0.08179045 0.9032147
## [1] "I would prefer other applications like Evernote, Google Docs, or MS Word."
##             2019         2020         2021
## 2020 0.350094298           NA           NA
## 2021 0.285699438 7.806553e-01           NA
## 2022 0.000694997 7.056524e-06 1.613306e-06
## [1] "The structure of the Notebook in Sections, Pages, and Subpages was easy to understand and use."
##      2019      2020      2021
## 2020    1        NA        NA
## 2021    1 1.0000000        NA
## 2022    1 0.9442328 0.9442328
## [1] "I would recommend to use OneNote for the ELN next year."
##             2019         2020       2021
## 2020 0.002215487           NA         NA
## 2021 0.172920608 1.962236e-01         NA
## 2022 0.228618042 6.958609e-05 0.01373391
## [1] "It overall allowed us to save time through the whole course."
##           2019       2020      2021
## 2020 0.3052495         NA        NA
## 2021 0.7158066 0.54233771        NA
## 2022 0.5564205 0.03337407 0.5423377
## [1] "It helped us in the preparation of the final version to be presented."
##           2019      2020       2021
## 2020 0.1701509        NA         NA
## 2021 0.1701509 0.9558176         NA
## 2022 0.7062676 0.0374604 0.03256321
## [1] "It made it easier working in groups."
##            2019         2020       2021
## 2020 0.01181064           NA         NA
## 2021 0.50976729 5.372195e-02         NA
## 2022 0.10713885 2.366656e-05 0.04984517
## [1] "It could be useful for other practical courses."
##            2019        2020      2021
## 2020 0.09691404          NA        NA
## 2021 0.55929640 0.598200546        NA
## 2022 0.59820055 0.009901279 0.1035011
## [1] "Synchronization issues hindered our teamworking."
##            2019      2020       2021
## 2020 1.00000000        NA         NA
## 2021 1.00000000 1.0000000         NA
## 2022 0.06488059 0.2322175 0.06488059
## [1] "Conflicts of versions were very frequent."
##            2019      2020      2021
## 2020 0.74604855        NA        NA
## 2021 0.74604855 0.9710615        NA
## 2022 0.03894251 0.5201819 0.5201819
## [1] "Access to the ELN was slow."
##           2019       2020 2021
## 2020 0.2325937         NA   NA
## 2021 0.7892961 0.02082084   NA
## 2022 1.0000000 0.10289719    1
## [1] "We missed several wordprocessing features."
##      2019 2020 2021
## 2020    1   NA   NA
## 2021    1    1   NA
## 2022    1    1    1
## [1] "Some features of the application did not work properly."
##            2019      2020 2021
## 2020 1.00000000        NA   NA
## 2021 0.11460584 0.1956151   NA
## 2022 0.05474198 0.1146058    1
## [1] "The template helped us to prepare the final version of the notebook."
##              2019         2020       2021
## 2020 7.618311e-01           NA         NA
## 2021 1.688823e-05 0.0004018526         NA
## 2022 3.999242e-01 0.7618310929 0.01161285
## [1] "The structure of the template matches the evaluation rubric."
##              2019         2020        2021
## 2020 1.0000000000           NA          NA
## 2021 0.0001986663 0.0009679592          NA
## 2022 1.0000000000 1.0000000000 0.006709094
## [1] "The structure of 4 independent notes helped to understand each step of the work in a research lab."
##           2019      2020      2021
## 2020 0.7663575        NA        NA
## 2021 0.2501597 0.9376949        NA
## 2022 0.9376949 0.7663575 0.2501597
## [1] "The word limit in some sections of the template is appropriate (1"
##      2019 2020 2021
## 2020    1   NA   NA
## 2021    1    1   NA
## 2022    1    1    1
## [1] "We had to modify some sections of the template to suit our experimental work."
##              2019      2020      2021
## 2020 3.220546e-06        NA        NA
## 2021 1.115448e-06 0.8425691        NA
## 2022 1.868463e-03 0.3594123 0.3594123
## [1] "The template will facilitate an even evaluation."
##              2019        2020        2021
## 2020 9.979675e-01          NA          NA
## 2021 6.709339e-05 0.003506002          NA
## 2022 1.000000e+00 1.000000000 0.001176378
## [1] "The use of an ELN with OneNote in this course will help us in future laboratory work (TFG, TFM, etc.)."
##           2019      2020      2021
## 2020 0.4321188        NA        NA
## 2021 0.4232123 1.0000000        NA
## 2022 1.0000000 0.4232123 0.3318217

Boxplots

Answers from each question are displayed in Boxplots by Year and significant differences from the overall answers are marked.

#update question nubers
#stack table and add question type (except open)
subdata.stack <- cbind(subdata[,1],stack(subdata[,2:72]))
names(subdata.stack) <- c("Curso","value","question")
#add categories
for (i in 1:nrow(subdata.stack)){
  subdata.stack[i,4] <- questions$Section[subdata.stack[i,3]==paste0("Q",questions$No.)]
}
names(subdata.stack) <- c("Curso","value","question","section")

pp <- list()
plot <- list()

#plot in groups by section


for (i in 1:length(levels(as.factor(subdata.stack$section)))){
  kkk <- subdata.stack[subdata.stack$section==levels(as.factor(subdata.stack$section))[i],]
  pp[[i]] <- ggboxplot(kkk, x = "Curso", y = "value",
          color = "Curso", fill = "Curso",palette = "jco",
          add = "jitter",alpha = 0.3, ylim = c(0, 6),
          facet.by = "question")
  plot[[i]] <- pp[[i]] + ggtitle(levels(as.factor(subdata.stack$section))[i]) + theme(axis.text.x = element_text(size = 15, angle=45,vjust=0.7),axis.text.y = element_text(size = 15),plot.title=element_text(hjust=0.5,face="italic")) + theme(legend.position="none") +
   stat_compare_means(label = "p.signif",method = "t.test", ref.group = ".all.", hide.ns = TRUE,label.y=5.5,size=6) 
}
plot[[1]]

plot[[2]]

plot[[3]]

plot[[4]]

plot[[5]]

plot[[6]]

plot[[7]]

plot[[8]]

plot[[9]]

Likert scale plots

Answers in Likert scales are represented by sections in the quiz.

General Methodology

#lickert
#change question names
tablita <- data.frame(matrix(NA,    # Create empty data frame
                          nrow = length(colnames(subdata)),
                          ncol = 2))
for (i in 2:length(colnames(subdata))){
  tablita[i-1,] <- cbind(colnames(subdata[i]),questions$Question[as.numeric(gsub("\\D", "",colnames(subdata[i])))])
  colnames(subdata)[i] <- tablita[i-1,2]
}

for (i in 2:72){
  subdata[,i] <- factor(subdata[,i])
}
#questions with 3 options
#General Methodology
subdata[,c(8:12)] <- lapply(subdata[,c(8:12)], function(x) factor(x, 
      labels = c("Traditional","Same","Open Question"))
  )
xlikgroup3a = likert(subdata[,c(8:12)], grouping = subdata$Curso)
plot(xlikgroup3a, type = "bar", centered = T) 

Activities Length

subdata[,c(29:35)] <- lapply(subdata[,c(29:35)], function(x) factor(x, 
      labels = c("Less","Fine","More"))
  )

xlikgroup3b = likert(subdata[,c(29:35)], grouping = subdata$Curso)

plot(xlikgroup3b, type = "bar", centered = T)

#title(main = "Activities Length", xlab = "X axis", ylab = "Y axis", cex.main = 4,   font.main = 3)
#legend("bottom",  c("Less","Fine","More"))

Equipment

subdata[,c(2:4)] <- lapply(subdata[,c(2:4)], function(x) factor(x, 
      labels = c("Not at all","Disagree","OK","Agree","Completely Agree"))
  )

xlikgroup5a = likert(subdata[,c(2:4)], grouping = subdata$Curso)
plot(xlikgroup5a, type = "bar", centered = T)

Length and Schedule

subdata[,c(5:7)] <- lapply(subdata[,c(5:7)], function(x) factor(x, 
      labels = c("Not at all","Disagree","OK","Agree","Completely Agree"))
  )
xlikgroup5b = likert(subdata[,c(5:7)], grouping = subdata$Curso)
plot(xlikgroup5b, type = "bar", centered = T)

Method Objectives

subdata[,c(13:21)] <- lapply(subdata[,c(13:21)], function(x) factor(x, 
      labels = c("Not at all","Disagree","OK","Agree","Completely Agree"))
  )
xlikgroup5c = likert(subdata[,c(13:21)], grouping = subdata$Curso)
plot(xlikgroup5c, type = "bar", centered = T)

Activities Interest

subdata[,c(22:28)] <- lapply(subdata[,c(22:28)], function(x) factor(x, 
      labels = c("Not at all","Disagree","OK","Agree","Completely Agree"))
  )

xlikgroup5d = likert(subdata[,c(22:28)], grouping = subdata$Curso)
plot(xlikgroup5d, type = "bar", centered = T)

Assessment

subdata[,c(36:41)] <- lapply(subdata[,c(36:41)], function(x) factor(x, 
      labels = c("Not at all","Disagree","OK","Agree","Completely Agree"))
  )

xlikgroup5e = likert(subdata[,c(36:41)], grouping = subdata$Curso)
plot(xlikgroup5e, type = "bar", centered = T)

Learning Objectives

subdata[,c(43:50)] <- lapply(subdata[,c(42:50)], function(x) factor(x, 
      labels = c("Not at all","Disagree","OK","Agree","Completely Agree"))
  )

xlikgroup5f = likert(subdata[,c(42:50)], grouping = subdata$Curso)
plot(xlikgroup5f, type = "bar", centered = T)

ELN

#subset to remove empty years
subdata[,c(51:72)] <- lapply(subdata[,c(51:72)], function(x) factor(x, 
      labels = c("Not at all","Disagree","OK","Agree","Completely Agree"))
  )
eln <- subset(subdata[,c(1,51:72)][subdata$Curso==2019|subdata$Curso==2020|subdata$Curso==2021|subdata$Curso==2022,])


xlikgroup5g = likert(eln[2:23], grouping = eln$Curso)
plot(xlikgroup5g, type = "bar", centered = T, title="ELN")

Session Info

sessionInfo()
## R version 4.2.2 (2022-10-31)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Big Sur ... 10.16
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] dplyr_1.1.0       reshape2_1.4.4    heatmaply_1.4.2   viridis_0.6.2    
##  [5] viridisLite_0.4.1 plotly_4.10.1     ggpubr_0.5.0      likert_1.3.5     
##  [9] xtable_1.8-4      corrplot_0.92     kableExtra_1.3.4  data.table_1.14.6
## [13] ggplot2_3.4.0    
## 
## loaded via a namespace (and not attached):
##  [1] httr_1.4.4         sass_0.4.5         tidyr_1.3.0        jsonlite_1.8.4    
##  [5] foreach_1.5.2      carData_3.0-5      bslib_0.4.2        assertthat_0.2.1  
##  [9] highr_0.10         yaml_2.3.7         pillar_1.8.1       backports_1.4.1   
## [13] lattice_0.20-45    glue_1.6.2         digest_0.6.31      RColorBrewer_1.1-3
## [17] ggsignif_0.6.4     rvest_1.0.3        colorspace_2.1-0   htmltools_0.5.4   
## [21] plyr_1.8.8         psych_2.2.9        pkgconfig_2.0.3    broom_1.0.3       
## [25] purrr_1.0.1        scales_1.2.1       webshot_0.5.4      svglite_2.1.1     
## [29] tibble_3.1.8       farver_2.1.1       generics_0.1.3     car_3.1-1         
## [33] cachem_1.0.6       withr_2.5.0        lazyeval_0.2.2     cli_3.6.0         
## [37] mnormt_2.1.1       magrittr_2.0.3     evaluate_0.20      fansi_1.0.4       
## [41] nlme_3.1-161       rstatix_0.7.2      xml2_1.3.3         tools_4.2.2       
## [45] registry_0.5-1     lifecycle_1.0.3    stringr_1.5.0      munsell_0.5.0     
## [49] ggsci_2.9          compiler_4.2.2     jquerylib_0.1.4    ca_0.71.1         
## [53] systemfonts_1.0.4  rlang_1.0.6        grid_4.2.2         iterators_1.0.14  
## [57] rstudioapi_0.14    htmlwidgets_1.6.1  labeling_0.4.2     rmarkdown_2.20    
## [61] gtable_0.3.1       codetools_0.2-18   abind_1.4-5        TSP_1.2-2         
## [65] R6_2.5.1           seriation_1.4.1    gridExtra_2.3      knitr_1.42        
## [69] fastmap_1.1.0      utf8_1.2.2         dendextend_1.16.0  stringi_1.7.12    
## [73] parallel_4.2.2     Rcpp_1.0.10        vctrs_0.5.2        tidyselect_1.2.0  
## [77] xfun_0.36